A high-performance image processing pipeline for Polony DNA re-sequencing

DNA Sequencing and re-sequencing are two fundamental tools in biological research, and applied to numerous such as genome mapping and genetic diseases related to DNA mutation. Polony DNA re-sequencing [POR06] is a modern high-throughput technique, which has been implemented in numerous systems, and the baseline software this project is based on is the image processing pipeline originally written for the Polonator G.007 machine [POL], which gathers relatively low-resolution images and extracts the relevant data to perform DNA re-sequencing operations. Due to the ongoing quest towards the cost reduction of re-sequencing large amounts of DNA material, for example the human genome, a new high-throughput parallel image processing pipeline has been designed to support a different selection of algorithms and to exploit SMP parallel computing systems, using a combination of pipeline, data-parallel and task-based parallelism patterns. The new software outperforms the original implementation in terms of total running time on a reference SMP system, while operating on 5.5x higher resolution images, as a result of both pipelining of the different processing stages on different processors and assigning multiple processors to the most computationally intensive stages, while overall load balancing amongst the stages is managed by a task-based task-stealing scheduler.